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ABSTRACT 


The  first  part  of  this  paper  provides  a  brief  tutorial  introduction 
of  the  Bayesian  Approach  to  identification  of  a  remotely  sensed  environ¬ 
ment.  The  second  part  describes  the  input  data  deck  setup  for  the  Fortran 
IV  program  which  has  been  written  to  implement  this  approach.  The  third 
part  describes  file  usage  and  subroutine  organization.  The  fourth  part 
provides  a  listing  of  the  program  with  a  simple  sample  data  set. 
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PART  I 


THE  BAYESIAN  APPROACH  TO  IDENTIFICATION 
OF  A  REMOTELY  SENSED  ENVIRONMENT 


I.  THE  BAYESIAN  APPROACH  TO  IDENTIFICATION 
OF  A  REMOTELY  SENSED  ENVIRONMENT 


Using  remote  sensors  we  can  make  measurements  of  an  environ¬ 
ment.  The  set  of  measurements  made  will  be  called  the  data  set.  Our 
Job  is  to  examine  the  data  set  in  order  to  identify  what  the  environment 
is  made  up  of:  our  problem  is  how  should  we  do  it?  In  what  follows 
we  describe  the  Bayesian  decision  approach  with  a  deterministic  decision 
rule. 

We  assume  that  distinct  boundaries  enclose  a  limited  environment, 
which  is  made  up  of  small-area  patches,  one  next  to  the  other.  The 
identification  of  the  environment  consists  of  identifying  each  small-area 
patch  within  one  category  of  a  given  set  of  categories.  We  assume  that 
such  an  identification  is  sensible  and  possible. 

In  order  to  make  any  identification  we  must  have  knowledge  con¬ 
cerning  which  kind  of  measurements  are  typical  measurements  of  the 
categories  we  wish  to  identify.  This  knowledge  is  succinctly  contained 
in  a  classification,  which  is  a  mapping,  associating  with  each  measure¬ 
ment  the  category  to  which  it  is  most  typical  -  given  a  specific  decision 
criterion.  Therefore,  if  we  are  to  identify  measurements  in  a  data  set 
we  must  have  a  classification. 

How  do  we  obtain  a  classification?  We  perform  an  information 
gathering  experiment.  From  the  population  of  all  environments ,  we 
sample  one  or  a  few  in  which  it  is  possible  to  identify  many  small-area 
patches  within  each  category  of  interest.  The  proportion  of  occurrence 
of  each  category  in  the  sampled  environment(s)  does  not  have  to  be 
representative  of  the  average  probability  of  occurrence  of  each  category 
in  the  entire  environmental  population.  However,  if  we  have  no  infor¬ 
mation  regarding  the  average  probability  of  occurrence  of  each  category 
in  the  environmental  population,  then  we  would  want  to  choose  the 
sampled  environment(s)  so  that  the  proportion  of  occurrence  of  each 
category  in  the  sampled  environment! s)  is  an  unbiased  estimate  of  the 


average  probability  of  occurrence  of  each  category  in  the  environmental 
population.  In  either  case,  the  small-area  patches  v/ithln  each  of  the 
sampled  environment(s)  do  have  to  be  representative  of  the  categories 
with  which  they  are  identified. 

With  each  of  our  sensors,  we  measure  each  small-area  patch  in 
the  chosen  environments.  From  photo-interpretation  or  field  studio 
the  environments  arc  examined  first  hand,  and  an  identification  of  each 
small-area  patch  is  made.  The  sequence  of  such  identifications  is 
called  the  "ground  truth  identification"  or  simply  "ground  truth".  It  is 
from  the  data  set  (the  sequence  of  measurements  )  and  the  ground  truth 
(the  sequence  of  identifications)  that  we  can  find  a  Bayes  classification. 
At  this  point  we  must  introduce  some  mathematical  notation. 

1/ 

Let  C  =  {c.}N  be  the  set  of  K  given  categories;  c.  is  the  symbol  used 
1  i=l 

for  the  i^  category.  We  suppose,  for  convenience,  that  each  sensor 
produces  only  one  number  for  each  measurement  it  makes  of  a  small- 
area  patch  We  suppose  further,  that  the  jth  sensor  must  produce  a 
number  belonging  to  its  range  set  Lj.  =  {  1^ ,  lj2»  •  ♦  • *  }  •  This 

supposition  is  fully  in  accord  with  reality,  since  the  output  of  any 
sensor  is  always  equivalent  to  a  pointer-reading  on  a  dial.  Pointer- 
readings  can  never  be  discerned  precisely,  and  are  thus  discerned 
approximately  to  third,  or  fourth,  or, . . .  ,  place  accuracy. 

Measurement  space  M  is  the  set  of  all  measurements  which  are 
possible  to  make  with  the  set  of  S  sensors.  M  is  conveniently  described 
as  the  cartesian  product  of  the  range  sets;  M  =  x  L2  x.  .  .x  Lg.  This 
is  the  set  of  measurements  which  contain  for  elements,  all  the  possible 
numbers  produced  by  sensor  one,  combined  with  all  the  possible  numbers 
produced  by  sensor  two, . . . ,  combined  with  all  the  possible  numbers 
produced  by  sensor  S.  For  convenience  we  number  the  measurements  in 

M;  M  *{m  ,  where  N  is  th_*  total  number  of  elements  in  measurement 

nn=  1 

space.  Finally  wo  must  provide  .1  goodness  criterion;  thus,  we  introduce 
a  gain  function  g.  g(oj,Cj)  is  our  economic  gain  if  we  identify  a  xneasure- 
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ment  as  belonging  within  the  i^  category  when  that  measurement  was 
made  of  a  small-area  patch  actually  belonging  within  the  category. 

We  have  already  mentioned  that  a  classification  is  a  mapping  or 
rule  which  associates  with  each  measurement  mR  in  M,  the  category  c^ 
to  which  it  is  most  typical  -  according  to  some  decision  criterion.  Our 
decision  criterion  is  economic;  "most  typical  to"  translates  to,  "that 
association  by  which  we,  on  the  average,  gain  the  most  economically". 
Therefore,  according  to  our  decision  crit  rlon,  we  can  Judge  each  possible 
classification.  That  classification  which  enables  us  to  gain  the  most, 
on  the  average,  is  the  classification  which  is  best;  it  is  that  classification 
which  we  wish  to  find. 

Let  us  now  examine  how  the  average  gain  may  be  calculated.  Let 

f  be  a  classification  mapping,  f  is  a  function  whose  domain  is  the  set 

M,  and  whose  range  is  the  set  C;  f:  M-*C.  For  each  element  m  e  M 

n 

the  function  associates  one  and  only  one  category  e  C  .  We  define  the 
characteristic  function  h^  for  f  as  follows:  for  every  m^  r  M,  C. 


h,(c.,m  )  =  1  if  and  only  if  f(m  )  -  c. 
fin  7  n  l 

0  otherwise 

In  other  words  h^Cj.m  )  is  1  if  and  only  if  the  classification  f  identifies 
the  measurement  as  belonging  within  the  category  Cj.  The  average 
gain  A  for  the  classification  f  is  easily  seen  to  be: 

K  K  N 

M)'  22  2  9(vci)hftuk  n-„)p(mj''i>p(ci) 

i*l  k=l  n=  1 


where  P(m  j c . )  is  the  conditional  probability  that  the  measurement  m 
n>  l  n 

will  be  made  of  a  small-area  patch  given  that  the  patch  belongs  within 
category  c^,  P\e^)  is  the  probability  that  any  small-area  patch  of  the 
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environments  in  the  population  belongs  within  category  c^,  and  g(cy,,c^) 
is  the  amount  gained  if  a  patch  which  actually  belongs  within  category 
c^  is  identified  within  category  c^. 

Of  the  four  terms  in  the  summation,  glc^c^)  is  specified  as  part 
of  the  identification  goodness  criteria,  h^c^m  )  is  defined  from  the 
classification  f,  P(mn|c^)  will  be  determined  from  the  data  gathered  in 
the  experiment,  and  P(c^)  is  an  additional  a  priori  probability  which  we 
will  have  to  specify.  Let  us  now  examine  in  detail  how  the  conditional 
probabilities  are  determined  from  the  experimental  data. 

The  data  set  is  a  sequence  D  of  R  measurements; 


The  ground  truth  corresponding  to  sequence  D  is  a  sequence  T  of  R  not 

necessarily  different  category  identifications;  T  =  ^  ,  c  , . . .  ,c  \  . 

rl  r2  rR' 

Lot  #  be  the  counting  measure.  #(D)  is  the  number  of  elements  in  the 

sequence  D;  thus,  #(D)  =  R.  A  sequence  is  really  a  function  whose 

domain  is  the  set  of  integers  I.  The  data  set  D  is  then  a  function  which 

associates  wtth  each  integer,  a  measurement;  D:  I  -*-M.  The  ground  truth 

1  is  also  a  function  and  it  associates  with  each  Integer  a  category; 

T;  I  -♦C.  D(7),  for  example,  is  then  Just  the  seventh  element  in  the 

sequence  D;  D(7)  =  rn  .  D  *(m)  is  the  set  of  all  integers  i  for  wh'ch 

r7  A  , 

D(i)  «  m.  The  statistic  P(m  c.)  estimating  P(m  c.)  is  defined  as 

n  i  i  n  *  * 


A  #(D_1(m jriT'l(c  ))  , 

P(m  |c  )  *  - - - - when  #(T  (c.))  t  0 

nl  1  nrhc,))  1 


-  0  otherwise 


I 

P(mnlCi)  is  num^cr  integftrs  which  are  associated  with  the  measure¬ 
ment  mn  in  the  sequence  D  and  with  the  category  Cj  in  the  sequence  T, 
divided  by  the  number  of  integers  associated  with  the  category  c^  in 
sequence  T.  Stated  simply,  Ptm^jc^  is  just  the  number  of  times  the 
measurement  was  made  of  a  small-area  patch  belonging  within 
category  c divided  by  the  number  of  times  a  small-a  -  patch  belong  id 
within  the  category  c,. 

The  a  priori  probabilities  Ptc^)  can  either  be  estimated  from  the 
sampled  data  set  (if  this  data  set  is  representative  of  the  population;  or 
from  our  foreknowledge  of  the  population  of  environments.  If  we  can 
assume  that  the  few  environments  we  have  chosen  to  sample  for  our 
experiment  are  representative  of  me  population,  then 

.  #(T_1(c,l) 

P(c ,)  = - i- 

R 

is  a  reasonable  estimate.  If  we  cannot  make  such  an  assumption  and  we 
believe  that  a  small-area  patch  is  just  as  likely  to  belong  within  one 

A 

category  as  within  another,  then  Pfcc)  -  1/K  is  a  reasonable  estimate. 

From  the  estimates  P(m  jo.)  and  P(c.)  wo  may  estimate  the 
average  gain  A  for  any  classification  f.  As  before  let  h^.  be  the 
characteristic  function  for  f. 


ht(c,  ,  m  )  -  i  if  and  only  it  f(m  )  -  c. 
f  k  tv  n  1 


0  otherwise 


K  K 


A 

Aft) 


HI  '  In,i( 


i  - 1  k-1  tr-l 


We  seek,  the  Bayes  c!as:.i?.e  uion  **  which  maximizes  A.  t*  is 
easily  detined.  Fur  each  inea.,ui<  meat  n.  ana  tor  any  classitlcation  f. 


there  will  be  one  and  only  one  category  e.  such  that  h,(c.,m  )  -  1 .  Ccn- 
a  /  i  J  ti 

slder  the  amount  a(Cj,mn)  gained  due  to  the  identification  of  measurement 

mn  as  belonging  within  category  c^. 

K 

a(Cj#mr)  =  ^g(cj<ci)P(mn|ci)P(ci) 
i=l 


A 

The  maximum  A(f)  is  certainly  achieved  if  for  each  measurement  m  , 

A  ^ 

f(mn)  =  Cj  where  c^  maximizes  a^m^).  Therefore  we  just  have  to 

compute  atCj.rn^)  for  j  =  1,  2,...,K  to  determine  which  category,  c^, 

maximizes  it.  Then  we  define  f*(m  )  “  c.. 

n'  j 

In  this  manner  we  can  define  how  to  best  identify  each  measurement 
which  actually  occurred  in  the  data  sequence  D.  However,  there  may  be 
many  measurements  in  measurement  space  M  which  did  not  occur  in  the 
data  sequence.  How  should  these  measurements  bo  identified  in  the 
classifica  tion?  Since  we  have  r.o  data  or  statistics  for  these  measurements 
it  seems  that  we  have  no  way  to  deal  with  them!  Here  we  must  draw 
upon  our  knowledge  of  the  structure  of  reality.  We  know  that  in  any 
environment  if  a  measurement  m  is  made  of  a  small-area  patch  belong.ng 
within  category  c^,  then  it  is  likely  to  make  measurements  m  +  6  for  other 
small-area  patches  which  also  belong  within  category  c^.  If  a  measure¬ 
ment  m  is  typical  of  category  c. ,  then  for  small  6,  m  +■  6  is  also  typical 
of  category  c^.  Similar  or  close  measurements  are  usually  associated  with 
similar  or  the  some  categories.  Thus  in  the  classification  we  can  identify 
a  measurement  m,  which  did  not  occur  in  the  data  sequence,  with  the 
category  associated  with  m',  its  nearest  neighbor. 

The  part  of  the  classification  f*  which  was  defined  by  moans  of 
the  statistics  generated  by  the  experiment  is  called  a  Bayes  Classification 
and  hence  the  name  "Bayesian  approach.  ”  The  part  of  the  classification 
which  is  not  Bayesian  is  said  to  be  defined  by  a  nearest  neighbor  search. 

Acknowledgement:  This  work  was  supported  by  Project  Themis  (USAP.TL 
Contact  DAAK02-68 -C-0089 ,  AKPA  order  No.  1079). 
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PART  II 


INPUT  DATA  DECK 


II.  INPUT  DATA  DECK 


The  data  for  this  program  arc  received  as  a  sequence  of  measure¬ 
ments  of  small- area  patches  or  objects  with  each  measurement  made  by  a 
sensor  or  set  of  sensors.  A  measurement  may  be,  for  example,  the  average 
backscatter  power  return  from  a  small-area  patch  at  Incidence  angles  5*,  10* 
15*,  20*,  30’,  40°,  50°,  and  60°.  In  this  case, each  measurement  has  8 
components.  The  patches  themselves  are  examined  and  identified  as  belong¬ 
ing  within  one  of  several  given  categories.  The  sequence  of  such  identifi¬ 
cations  is  called  the  "ground  truth  identification"  or  simply  the  "ground 
truth."  The  Bayes  program  can  determine  a  Bayes  classification  of  measure¬ 
ment  space,  based  on  the  data  and  the  grc and  truth  lor  the  data.  Once  a 
classification  is  determined,  the  Bayes  program  can  identify  each  measure¬ 
ment  within  a  sequence  of  meat  urements .  This  identification  is  done  by  a 
nearest- neighbor  search. 

The  input  deck  is  organized  into  four  sections;  title,  program  options 
parameter  cards,  and  format  and  data. 

I.  Title 

A)  The  title  section  consists  of  a  single  card  specifying  the 
name  of  the  data.  The  title  may  begin  in  column  one  and 
continue  through  column  eighty. 

II.  Program  Options 

A)  The  program  option  section  consists  of  two  cards,  the  first 
card  specifying  all  the  input  options  and  the  second  card 
specifying  all  the  output  options. 

B)  Each  option  is  a  six-character  abbreviation  or  code. 

C)  The*  options  start  in  column  16,  are  separated  only  by 
commas  (no  embedding  blanks)  and  may  appear  in  any 
order. 

D)  Only  input  options  may  appear  on  the  input  card  and  only 
output  optical;;  may  appear  on  the  output  card. 

1)  The  input  •  <pt  I*  >ns  are;  PiiOPTS,  CORPTS.  FETING  , 

FA  m;RN  .  IHAGON.  Hl.FNHF.  ABSQNT. 

2)  The  output  options  are;  AI.TICH.  STDPNT.  DPUNCH . 
PHOUT1.  l'l!OUT2,  TERMNE. 


III.  Parameters 

A)  The  parameter  section  consists  of  cards,  the  number  of 
which  varies  with  the  options  chosen. 

B)  There  are  seven  basic  types  of  information  which  can  pos¬ 
sibly  appear  in  the  parameter  section:  gain  matrix,  dimen¬ 
sionality  of  measurements,  number  of  measurements  in  the 
data  set,  number  of  categories  in  the  classification, 
display  size,  number  of  levels  to  which  the  measurements 
will  be  quantized,  and  means  of  estimating  the  a  priori 
probability  distribution. 

IV.  Format  and  Data 

A)  There  are  two  ways  to  organize  the  format  and  data:  photo¬ 
graphic  form  and  corresponding- point  form.  Depending  on 
the  options  chosen,  the  ground  truth  identification  and  its 
format  may  or  may  not  be  present.  One  and  only  one  of  the 
two  forms  must  be  specified:  otherwise,  an  error  message 
and  termination  of  the  job  will  result. 

1)  In  the  photographic  form  the  data  are  organized  as 

follows: 

a)  format  for  identification  (if  any) 

b)  identification  (if  any)  for  measurement  one,  measure¬ 
ment  two,  ...» meas  moment  N  . 

c)  format  for  measurements 

d)  component  one,  measurement  one;  component  one, 
measurement  two; . .  ..-component  one,  measurement 
N!  component  two,  measurement  one;  component 
two,  measurement  two; . .  ..‘component  two,  measure¬ 
ment  N; ..  .component:  M,  measurement  one;  compo¬ 
nent  M,  measurement  two; . . .  .‘Component  M,  mea¬ 
surement  v . 

2)  In  the  corresponding- point  form  the  data  are  organized 

as  follows: 

a)  format  for  identification  (if  any)  and  measurements 
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b)  identification  (if  any)  for  measurement  one;  compo¬ 
nent  one,  measurement  one;  component  two,  mea¬ 
surement  one;...;  component  measurement  one: 
identification  (if  any)  for  measurement  two;  compo¬ 
nent  one,  measurement  two;  component  two,  mea¬ 
surement  two;...  component  M,  measurement  two:., 
identification  (if  any)  for  measurement  N;  component 
one,  measurement  N;  component  two,  measurement 
N;...  component  M,  measurement  N. 


We  now  describe  the  options. 

I.  Input  Options 

A)  PHOPTS  --  is  the  abbreviation  for  photographic  form,  and 
is  used  when  the  data  are  in  that  form. 

B)  CORPTS  —  is  the  abbreviation  for  corresponding  point  form, 
and  is  used  when  the  data  are  in  that  form. 

C)  FLUNG  --  is  the  abbreviation  for  floating  point,  and  is 
used  when  the  data  are  punched  on  cards  in  floating-point 
form.  It  is  not  used  when  the  data  are  punched  on  cards 
in  integer  form. 

D}  PATERN  --  is  the  abbreviation  for  pattern  classification  by 
Bayes'  strategy.  Use  of  this  option  will  output  a  probability 
matrix  where  element  (i ,  j)  is  the  conditional  probability 
that  a  measurement  which  was  identified  as  within  the  i^ 
ground  truth  category  is  identified  in  the  classification  as 
within  die  j^1  category.  A  punched  dock  of  the  compacted 
quantized  measurements  with  their  identifications  in  the 
Bayes'  classification  will  also  be  produced. 

E)  DLAGON  --  is  the  ubhieviution  for  diagonal  gain  matrix 
with  ones  on  the  diagonal .  Specification  of  DIAGON  will 
internally  generate  an  identity  matrix  for  the  gain  matrix. 
This  relieves  t! i- *  user  of  the  need  to  supply  the  appropriate 
cards  in  the  parameter  section. 


F)  HI.FNHF  --  is  the  abbreviation  for  half  and  naif.  Speci¬ 
fication  of  HLFNHF  will  divide  the  data  into  halves:  even 
and  odd  points.  The  first,  third,..  .,  data  points  are  used 
to  construct  a  Bayes’  classification,  and  the  second,  fourth, 
. . .  ,data  points  are  identified  on  the  basis  of  the  classifica¬ 
tion. 

G)  ABSQNT  --  is  the  abbreviation  for  absolute  maximum 
quantization.  Specification  of  ABSQNT  will  quantize  the 
measurements  by  determining  the  minimum  and  maximum 
values  occurring  among  ali  the  components,  normalizing 
each  component  by  subtracting  ties  minimum,  dividing  by 
this  maximum  minus  this  minimum,  and  multiplying  by 
the  number  of  quantized  levels  desired.  If  ABSQNT  is  not 
specified,  the  program  finds  the  maximum  and  minimum  for 
each  component,  and  normalizes  component  by  component. 

II.  Output  Options 

A)  ALTPCH  —  is  the  abbreviation  for  alternate  punch.  Speci¬ 
fication  of  ALTPCH  produces  a  punched  quantized  data 
deck  in  the  alternate  form.  If  the  data  are  in  photographic 
form,  the  punched  deck  will  be  in  corresponding- point 
foim.  If  the  data  are  in  corresponding- point,  the  punched 
deck  will  be  in  photographic- point  form. 

B)  STDPNT  --  is  the  abbreviation  for  standard  identification 
print  out.  Specification  of  STDPNT  will  print  out  the  ground 
truth  identification. 

C)  DPUNCH  —  is  the  aobreviation  for  punched  deck.  Specifi¬ 
cation  of  DPUNCH  will  produce  a  punched  quantized  data 
deck  in  the  same  form  as  the  input  data. 

D)  PHOUTl  --  is  the  abbreviation  for  photograph  output. 
Specification  of  PHOUTl  will  identify  data  according  to 

a  given  classification .  The  classification  can  be  internally 
generated  by  the  Bayes  routine  or  it  can  be  externally 
supplied  in  the  data  deck.  Identification  is  done  by  a 
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table  look-up  procedure.  If  the  quantized  measurement 
cannot  be  found  in  the  classification,  a  nearest- neighbor 
search  is  initiated,  and  the  measurement  is  identified 
with  the  same  identification  as  the  closest  measurement 
to  it  in  the  classification. 

If  the  input  include?  ground  truth  identification  for  a  data 
set  and  the  data  are  further  identified  relative  to  a  classi¬ 
fication  (by  specification  of  PHOUT1),  then  a  contingency 
table  of  the  ground  truth,  identification  versus  the  classifi¬ 
cation  identification  will  be  printed  out.  The  (i,j)*^  ele¬ 
ment  of  the  table  is  the  number  of  measurements  which 
were  identified  in  the  ground  truth  category  and  classified 
in  the  ground  truth  category. 

E)  PHOUTD  --  is  the  abbreviation  for  photograph  output  deck. 
Specification  of  PHOUTD  will  produce  a  punched  deck  of 
the  identification  which  the  photograph  output  routine 
determined.  PHOUTD  can  only  bo  specified  if  PHOUT1 

is  specified. 

F)  TERMNI.  is  the  abbreviation  for  remote  terminal  format. 
Specification  of  TEP.MNL  will  format  all  printed  output 
from  the  Bayes  program  so  that  each  i.us  no  more 
than  seventy  columns,  if  TERMNL  is  not  specified,  each 
line  is  print.nl  with  one  hundred  thirty  columns. 

The  generality  of  the  program  requires  that  the  input  parameters  be 
quite  variable,  depending  on  the  type  of  data  to  be  processed  and  the  options 
desired.  Fcr  example,  the  PATERN  option  may  or  may  not  require  the  gain 
matrix  as  input  depending  on  whether  oj  not  1)IA  '.OH  was  also  specified. 

To  facilitate  setting  up  the  input  date  deck,  a  flow  chart  is  illustrated  in 
Figure  la  and  lb. 

The  flow  chart  contains  two  symbols.  The  first  symbol  is  the 
elongated  circle  (rectangle  with  i  >un  led  edges),  which  asks  the  question 
printed  in  the  circle  and  requires  a  "true"  or  "false"  answer.  Depending 
upon  the  user’s  answer,  on*:  branch  is  ch  >  n-n  irom  the  bottom  of  the  circle. 


The  second  symbol  is  the  rectangle,  which  usually  signals  the 
addition  of  a  single  data  card.  Data  cards  are  added  when  rectangles  are 
encountered  in  the  flow  chart.  Two  exceptions  to  the  "one- rectangle,  one- 
data-card"  rule  are  the  gain  matrix  and  the  data  itself.  If  the  gain  matrix 
is  required  (PATERN  is  specified  and  DIAGON  is  not  specified),  the  user 
must  supply  all  the  elements  for  the  matrix.  This  will  take  up  more  than 
one  card  if  there  are  more  than  two  ground  truth  identification  groups.  Simi¬ 
larly,  if  there  are  more  than  eight  measurements  in  the  data  set,  the  data 
deck  will  have  more  than  one  card. 

The  final  control  card  is  the  STOP  card  and  is  the  last  card  in  the 
input  deck.  The  four- character  word  STOP  is  punched  in  columns  one 
through  four.  The  program  is  designed  to  handle  more  than  one  data  set  per 
run,  and  the  user  may  place  behind  the  first  (or  second,  or  third,  etc.)  data 
set  the  NAME  of  the  next  data  set,  the  proper  /INPUT  and  /OUTPUT  cards, 
and  the  other  necessary  parameter  cards  as  specified  by  the  flow  chart.  The 
STOP  card  is  placed  after  the  final  set  of  data,  and  informs  the  system  to 
terminate  the  job.  Excluding  the  STOP  card  or  misplacing  any  control  cards 
will  cause  a  read  error  and  termination  of  the  jcb. 


FLOW  CHART  FOR  SETTING  UP  INPUT  DATA  DECK 
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PART  III 


FILE  USAGE  AND  SUBROUTINE  ORGANIZATION 


III.  FILE  USAGE  AND  SUBROUTINE  ORGANIZATION 


Blank  common  storage  carries  al)  problem  parameters  and  user  options 
(CORPTS,  PHOPTS,  etc.),  as  well  as  providing  a  24,000-word  scratch  area. 
Many  routines  in  different  links  require  such  parameters  as  the  number  of 
dimensions  of  the  current  problem,  the  number  of  points  being  processed, 
and  the  number  of  ground  truth  categories. 

All  other  communications  between  links  are  handled  by  tape,  disc, 
or  drum  files.  The  program  requires  nine  files  —  01  ,  02,  03,  04,  09,  10, 

11,  20,  21  —  as  well  as  the  input  (05),  output  (06),  and  punch  (43)  files 
normally  used  in  FORTRAN.  Figure  2  describes  file  usage,  and  Figure  3 
illustrates  how  much  storage  is  needed  on  each  of  the  files. 

The  Bayes  program,  as  mentioned  before,  requires  36,000  words  of 
storage  in  the  computer  core,  of  which  25,000  may  be  shared  during  loading. 
The  program  has  been  observed  to  process  350  sixteen- dimensional  data 
points  in  less  than  ten  minutes'  processor  time. 
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FILE 

NUMBER  OF  WORDS  NEEDED 

01 

Total  points 

02 

Total  points 

03 

(Total  Points)  *  #  dimensions 

04 

09 

(Total  Points)  *  #  dimensions 

10 

2*{total  points) 

11 

l/2*(total  point) 

20 

2*(#  unique  n- tuples  in  data) 

21 

(Total  Points)  *  #  dimensions 

figure  3.  Storage  Requirements  for  the  Tape  or  Disc  Files 
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The  program  organization  is  briefly  described  below.  Listed  first 
are  the  mainline  and  CHNXT,  the  system  supervisor,  followed  by  the  six- 
character  link  names  used  during  overlay  processing.  Under  each  link 
name  are  listed  the  routines  contained  in  the  link.  Link  names  containing 
the  alphabetic  character  "A"  refer  to  links  essential  for  proper  data  pro¬ 
cessing:  link  names  containing  the  alphabetic  character  " B"  refer  to  links 
which  outpu'.  intermediate  calculations,  but  which  do  not  contribute  to  the 
overall  program  results. 

LINKB2 

. (mainline)  --  reserves  all  common  storage,  provides  entry 

point  to  program,  and  contains  comment  cards  stating  program  para¬ 
meters  and  input  deck  setup. 

CHNXT  —  a  small  resident  program  used  to  control  the  entry  and 
exits  of  the  different  links.  This  routine  must  comply  with  the 
overlay  rules  of  the  operating  system  in  use. 


LINKAi 

MAIN  —  parameter  input  and  selection  is  accomplished  in  this 
routine.  Also,  if  the  photo  classifications  and  the  gain  matrix  must 
be  read  in,  it  is  done  in  MAIN. 

FORM  —  selects  and  places  in  common  all  formats  affected  by 
terminal  or  non- terminal  use. 


LINKA2 

INPUT  --  performs  data  input,  saves  every  even  point  for  later  pro¬ 
cessing  if  desired,  and  changes  data  to  alternate  form  If  called  for. 

FPR-FPR1  --  searches  data  for  the  maximum  and  minimum  points  so 
that  proper  quantization  and  shifting  may  be  done  in  INPUT. 

DEF  --  defines  the  single- character  symbols  to  be  used  in  the 
classification. 

Id 


TRANE  —  prints  a  cross-reference  between  the  single- character 
symbols  used  in  the  program  and  the  original  symbols. 

CHANGE  —  an  assembly  language  routine  which  creates  a  suitable 
output  format  for  the  data. 


LINKA3 

PATERN  —  Bayes  program 

OUTP  —  prints  out  results  of  PATERN  in  eye-appealing  format. 

DECSON  —  selects  and  assigns  to  each  n- tuple  the  proper  classi¬ 
fication  according  to  Bayes  theory. 


LIN  KB  3 

DMP  —  used  for  debugging;  prints  out  n- tuples  vs.  categories 
and  n- tuples  vs.  classification. 


LINKA4 

PLOTER  —  classifies  and  plots  input  data  according  to  n- tuple 
classifications. 

SEARCH  —  searches  a  list  of  n- tuples  to  find  the  list  element  which 
is  closest  in  distance  to  another  n- tuple. 

IDIST  —  calculates  n-dimensional  space  distances. 
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PART  IV 


EXAMPLE  PROBLEM  AND  PROGRAM  LISTING 


IV.  EXAMPLE  PROBLEM 


Suppose  the  problem  is  to  input  a  set  of  data  In  photographic  form, 
punch  out  the  data  in  the  alternate  form,  print  the  data  identification,  and 
then  classify  a  new  set  of  data.  Let  there  be  nine  points  in  the  horizontal 
direction  and  ten  points  in  the  vertical  direction  for  the  first  data  set.  The 
data  appear  in  Figure  4a.  For  the  first  part  of  the  problem,  the  program  must 
produce  an  alternate  data  deck,  a  print-out  of  the  identification,  and  a  Bayes* 
classification  based  on  the  first  set  of  input  data. 

The  first  card  is  the  title  card.  The  next  two  cards  specify  the 
input  and  output  options  needed.  The  options  start  in  column  IS  and  are 
separated  by  commas.  The  data  are  in  photographic  form,  so  the  input 
card  is: 

/INPUT  PHOPTS ,  PATERN 

The  output  card  is: 

/OUTPUT  ALTPCH.STDPNT 

The  parameter  card  follows.  From  the  flow  chart  we  see  that  the  fourth  card 
must  specify  the  number  of  dimensions  per  measurement  (number  of  photo¬ 
graphs),  total  number  of  measurements,  number  of  quantized  levels,  and 
number  of  ground  truth  categories.  In  our  example  the  number  of  photographs 
is  two  and  the  number  of  measurements  is  ninety.  We  wish  to  quantize  the 
data  to  ten  levels  and  there  are  two  ground  truth  categories.  The  fourth 
card  thus  appears  as: 

2  90  10  2 

Since  PHOUT1  is  not  specified  and  STDPNT  is,  the  fourth  card  must  specify 
the  photograph  size,  which  is  nine  points  horizontal  by  ten  vertical.  The 
fifth  card  appears  as: 

9  10 
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111151111 

111151111 

111151131 

111151111 

555555555 

555555555 

111151111 

111151)11 

111151111 

1111511)1 

111151111 

Photo  1  for  Test  1 


222262222 

222262222 

222262222 

222262222 

333333333 

333333333 

222262222 

222262222 

222262222 

222262222 

Photo  2  (or  Test  1 


51115 

15151 

11511 

15151 

51115 


Photo  1  for  Test  2 


32223 

26262 

22322 

23232 

62226 


Photo  2  for  Test  2 


Figure  4a.  Data  for  Test  1 


Figure  *1b.  Data  for  Test  2 
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Since  PATERN  is  specified  and  DIAGCN  is  not,  the  user  must  supply 
the  gain  matrix.  Suppose  we  choose  a  gain  matrix  where  we  gain  ten  for  a 
correct  decision  and  lose  five  for  an  incorrect  decision,  as  illustrated  in 
Figure  5. 


10  -5 
-5  10 

Figure  5  .  Gain  Matrix 
The  sixth  card,  specifying  the  above  matrix,  is 
10.  -5.  -5.  10. 

We  must  now  indicate  how  the  a  priori  probability  distribution  is  estimated. 
We  choose  to  suppose  that  each  identification  group  has  equal  probability. 
Therefore  NORMALIZED  is  specified  on  the  next  card, 

NORMALIZED 


The  data  are  in  photographic  form,  so  the  format  for  the  ground  truth  identi¬ 
fication  must  come  next.  In  our  example  this  would  be: 

(9A1) . 

After  the  identification  format,  the  identification  itself  comes: 

AAAABAAAA 
A AAA BA AAA 
AAAABAAAA 
AAAABAAAA 
BBBBBBBBB 
BBBBBBBBB 
AAAABAAAA 
AAAABAAAA 
AAAABAAAA 
AAAABAAAA 
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Finally  we  reach  the  format  for  the  data,  the  data  itself,  and  the  STOP  card. 
In  our  example  these  appear  as: 


(911) 

111151  111 
1 11151  111 
111151111 
111151111 
555555555 
555555555 
111151111 
111151111 
111151111 
111151111 
111151111 
222262222 
222262222 
222262222 
222262222 
333333333 
333333333 
222262222 
222262222 
222262222 
222262222 
STOP 

The  input  data  deck  is  illustrated  in  Figure  5. 

At  this  point  we  must  run  a  job  with  the  input  deck  as  shown  in  Figure 
6  .  The  job  produces  a  punched  deck  of  tne  Bayes  classification.  We  must 
now  identify  a  new  set  of  data  which  is  illustrated  in  Figure  4b.  This  is 
done  by  a  separate  job.  The  first  card  is,  as  usual,  the  title  card.  The 
second  and  third  cards  are  tht.  input  and  output  option  cards.  The  new  set 
of  data  is  in  photographic  form,  and  we  wish  to  have  a  print- out  of  the  iden¬ 
tification  for  it,  based  on  the  classification  of  the  previous  job.  The  next 
cards  thus  appear: 


/INPUT  PHOPTS 

/OUTPUT  FHOUTi 


There  are  two  dimensions,  twenty-five  measurements,  ten  quantized  levels, 
and  two  identification  groups.  The  fourth  card  is: 


23 


2 


25 


10 


2 


Since  PHOUT1  is  specified,  the  next  card  must  indicate  the  photographic 
size  which  is,  in  our  example,  five  points  horizontal  by  five  vertical.  The 
next  card  is  thus: 


5  5 


The  number  of  quantized  measurements  in  the  classification  and  the  Bayes 
classification  itself  come  next.  These  cards  were  obtained  from  the  output 
of  the  previous  job. 

For  our  example,  they  are: 

3 

65.02  35.02  21.01 


All  that  now  remains  is  the  format  for  the  data,  the  data  itself,  and  the 
STOP  card. 


(511) 

51115 

15151 

11511 

15151 

51115 

32223 

26262 

22322 

23232 

62226 

STOP 


The  input  data  deck  is  illustrated  in  figure  7  . 
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Figure  6.  Input  Data  Deck  for  First  Example 
Problem 
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? 
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C0MMON  “P)itfOHHTi(i2) 

0 
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COmmok-  r 

ft 
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ft 
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7 
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ft 
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ft 
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ft 
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U»1 

0 
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1 1 1  ■» 

ft 

15 

2 

16 

NNaN 

0 
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3 

*7 

1  CALL  mnamF 

ft 

17 

4 

16 

k«nm 

ft 

10 

5 

19 

tf  IK.LT.0)  GO  TO  9 

ft 

19 

6 

20 

LIAEiO 

ft 

20 

9 

21 

Ka6 

ft 

?i 

10 

22 

NS«25*?: 

ft 

27 

11 

23 

Tf  (NS.EO.ft)  GO  To  3 

0 

23 

12 

2  4 
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ft 

24 
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25 
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ft 
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20 

7  UNITE  (0.15) 

ft 
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ft 
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20 
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79 

WRITE  <0. f MT12 ) 
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33 
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33 

36 

34 
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ft 

35 

30 

36 

If  (44-LINF.LE.0)  GO  TO  5 

ft 

30 

44 

37 
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0 

37 
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30 

If  U.UE.C)  GO  TO  5 

ft 

30 

40 

39 
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39 
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41 
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55 

42 
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43 

60 

44 
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n 
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ft 

57 

0* 

50 
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n 

58 

60 

59 

WRITE  (0.16) 

n 

59 

69 

60 
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ft 

60 

91 

61 
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ft 

01 

93 

*2 
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ft 

6? 

94 

63 
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70 
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o 

70 

115 

71 

WRITE  ( 6 «f MT&4 )  IDlU) 
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o 
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GO  TO  13 
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R*379  01  (>5-?N-49  21.004 


ChDFx  EXECUTION  REPORT 


*  FORTRAN  MECK.COf'OK  SEAROH 

SUBROUTINE  SF»°CW  (KPT.KLAS.KK.SYMROL.^.MP.K.LFVE-) 

DIMENSION  )PT(?),  KL AS  (?)  I  S»('4)  •  I 

r i «fc»js i o*y  )P(ioo) 

DIMENSION  l |ST(2)  •  4 

COMMON  / 1 S T T/  1ST  »  » 

DAT*  ST/lHA,lHP,tMC.]Mn,lME,lHT,tMO,lMh.tm.lMJ,ll«.lHt,lM»,1M«(,lt*  *  6 

10. IMP. 1H0.1 wP.1hS.1hT, 1MD.1MV. ImA.IMX, 1MY, 1m2, 1M1, 1M7.1M3. 1M4, IMS,  •  ? 

21M4.1M7.1PC. 1H9, 1H0/  •  S 

K  IS  the  CATEGORY  number 
SYMBOL  IS  THE  CATEGORY  CODE 
KK  IS  the  COMPACTED  NTUPlE 
J2  IS  UPPER  LIMIT  E OR  KPT  AND  K.AS  ARRAYS 
LOOK  F0S  COMPACTED  N-TUPlE  KK  IN  LIST 


KS*0  »  * 

KL*J?*1  *  10 

KTRY»(KSCKL )/2  P  11 

1  CONTINUE 

IE(KK-KET(KVRY1)  3.4.2 

2  IE  ( (KL-KS) .lE.ii  go  to  s  *  IS 

KL*KtRY  E  14 

KTRY«(KL«KS»1)E2  »  1* 

GO  TO  1  »  IS 

3  IE  (KL-KS. LF.1>  GO  TO  S  P  1* 

KS*KTRY  b  IS 

KTRYs(KL*KS)/2  *  1» 

GO  TO  1  P  20 

4  KlKLAS(KTRY)  P  21 

SYM80L*SY(K|  p  22 

RETURN  »  23 

5  Continue  »  2< 


C  KK  CANNOT  hE  FOUND  DO  NEAREST  NEIGHBOR  SEARCH 


II*  IMST(KK,KPT(1),NP.  LEVEL! 

L  *  1 

KP( 1 ) *1 

00  7  J*2,U?  p  20 

12* IDIST(KK,KPT( J) . NP. LEVEL ) 

IE  M1.LT.  12)  GO  TO  7  P  30 

IE  Ml.NE.  12)  Go  TO  4  »  31 

I E  ( i_  .  GT  .  99  )  GO  TO  7 

L*L«T  P  32 

KP(L)*J  »  3* 

GO  TO  7  »  35 

4  CONTINUE  *  34 

11*1?  “  37 

L*1 

KP(1  )*J 

7  CONTINUE  •  40 

X*BCM<IST>  »  44 

K*X.ELOAT(L-l)*l.» 


C  KP  IS  The  array  containing  the  INDEXES  ro»  ALL  THOSE  POINTS  IN 

C  The  KPT  ARRAY  WHICH  ARE  CLOSEST  TO  KK 

C  L  IT  the  UPPER  LIMIT  or  K» 

C  PE  WILL  CPOOSE  ONE  POINT  FROM  THE  KP  ARRaT  AT  RaNCOM  AND 

C  lOENT'EY  KK  WITH  THE  CATEGO»T  O’  THE  POINT  in  KPT  ASSOCIATED  u I TM 

c  the  randoplt  chosen  one 


K  *  KP ( K  ) 

3  K.KLAS(K)  P  46 

SymROL*St(K>  »  47 

PETyPN  p  4  S 

END  »  4». 


1*379  01  05-25-49  21.004  CHDSX  EXECUTION  REPORT 

|  FORTRAN  NOECK.COMOK  IDIST 

function  icistiko.kp.np, level) 

i J i st  «o 

KK*KG 

LL*KP 

NP1 *NP-1 

00  l  J»1.NP1 

K*KK/LFVFL 

L1*KK-K«LEVEL 

IiLL'LFVFL 

L2*LL-IpLEVEI 

KK*K 

LL*1 

1  IO'Sli|OIST»(Ll  -L2)p(L1-i.?> 

IOIST»IOIST*CK-I 1»(K-I » 

RETURN 

END 
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TEST  DATA 


iable  table 


PLOT  SYMBOLS 

vs  input  symbols 

A 

A 

B 

B 

TEST  DATA 

training  regions 

ABBABRRABA 

ABBABBBABA 

ABBA8PBAB* 

AB6A8BBABA 

A8BABBBA8A 

abbabbbaba 

abbabbbaba 

ABBA8BBAB* 

ABBABBBABA 

abbabbbaba 

ABBABBBABA 

ABBA8BBABA 

ABBABBBABA 

abbabbbaba 

abbabbbaba 

abbabbbaba 

abbabbbaba 

ABBABBBABA 

ABBABBBABA 

abbabbbaba 

A8BA8B8A8A 

ABBABBBABA 

abbabbbaba 

ABBABBBABA 

ABBABBBABA 

abbabbbaba 

abbabbbaba 

A8BA8B8A8A 

ABBABBBABA 

abbabbbaba 

ABBABB8A8A 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

abbabbbaba 

abbabbbaba 

abbabbbaba 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABBBABA 

ABBABRRABA 

ABBABBBABA 

ABBABRRABA 

abbabbbaba 

ABBABBBABA 

ABBABBBABA 

abbabbbaba 

ABBABBBABA 


ABBABBBABA 

ABBABBBABA 

iBBABRBABA 

abbabbbaba 

ABBABRBAB* 

ABBABRRABA 

ABBABBBABA 

abbabbbaba 

ABBA0RBA8A 

ABBABRRABA 

ABBABBBABA 

abbabbbaba 

ABBABBBABA 

ABBABhBABA 

abbabbbaba 

ABBABRRABA 

ABBA6R0ABA 

ABBABBBABA 

ABBABBBABA 

ABBABRRABA 

ABBABBBABA 

ABBABRBAB* 

ABBABRRABA 

ABBABRRABA 

ABBABRRABA 

ABBA0R0ABA 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAARBB 

AAAAAAAB8B 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAARBB 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAARBB 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAARBB 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAARBB 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAABBR 

AAAAAAARBB 
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TRAINING  REGIONS  NO. levels  no.  rhotos 
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TEST  04 T  4 


CL  » SS  I »■  IC»Mom 

4RB4B*R«B4 
4BR40tRR0  A 

*BR aBbS6B a 

ABBAR’RURA 
ABB ARJDP0J 

4H04BPRB04 

ABRA0MRU0A 

40040O0R04 

ABRA8Bnr,BA 

ABB 40R0H04 

ABBA0b0b0A 

4B040«qp04 

400 ABwRR0 A 

ABBiB^BBBi 

A0RA0n(|R0A 

A00AB6Pb0A 

AB0A0U0R0A 

400  4  0° RR0 4 

A0R A0B0M0 A 

ABB  *£B9H0  * 

4RBA0»RRRA 

ARB  *BBBNB  A 

AR0A0°Rn0 A 

AR0  * 0BRR0  A 

AB0  A0BRB0  A 

4O0A0BBH9 A 

A00  A  0RRRR  A 

4BRA0P9B0A 

A08A0PRP0.1 

ARR A0R0RRA 

Ad0  A  0O0H0  A 

APR A0DBPB* 

ABB  A0BRRQ  A 

ARR ABQRP0A 

APRA0A0O0A 

abrabprhba 

A00A0RRH0A 

ARB  A0B  0R04 

A60A0HRR0A 

A0R A0BSB0A 

ARR  A0®RH0  A 

ABRa0bBbRA 

AB0A0RBR0A 

ARRaBbBR0a 

ABB  A0BRM0  A 
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